Neural Networks: Algorithms and Special Architectures

The paper is focused on neural networks, their learning algorithms, special architecture and SVM. General learning rule as a function of the incoming signals is discussed. Other learning rules such as Hebbian learning, delta learning, perceptron learning, Least Mean Square (LMS) learning, Winner Take All (WTA) learning are presented as a derivation of the general learning rule. Architecture specific learning algorithms for cascade correlation networks, functional link networks, counterpropagation networks and Radial Basis Function (RBF) networks are described. Case study has been done for neural network based identification and control for temperature control of a water bath. Support Vector Machines (SVMs) an extremely powerful method of deriving efficient models for multidimensional function approximation and classification have been discussed. Introduction Artificial neural networks are systems that are deliberately constructed to make use of some organizational principles resembling those of the human brain. They represent the promising new generation of information processing systems. Artificial neural networks (ANNs) have a large number of highly interconnected processing elements (nodes or units) that usually operate in parallel and are configured in regular architectures. The collective behavior of an ANN, like a human brain, demonstrates the ability to learn, recall, and generalize from training patterns or data. ANNs are inspired by modeling network of real (biological) neurons in the brain. A survey [1],[2],[3],[4],[5],[6],[7],[8],[9],[10],[11],[12],[13] on neural networks learning methods and special feedback architectures have been done. Single-layer and multilayer feedforward networks and their associated supervised learning rules have 176 Bharat Bhushan and Madhusudan Singh been described. Case study on temperature control of water bath using feedforward NN have been done. Various learning algorithms like Hebbian learning rule, Correlation learning rule, Instar learning rule, Winner Takes All, Outstar learning rule, Perceptron learning rule, Widrow-Hoff learning rule, Delta learning rule and Error Backpropagation learning are discussed. Special feedforward architectures like Functional link networks, feedforward version of the counterpropagation network, LVQ learning vector quantization, WTA architecture, Cascade Correlation architecture and RBF-Radial basis function networks have been discussed. Both the multiplayer perceptron and the radial basis network are based on the popular learning paradigm of error-correction learning. The synaptic weights of these networks are adjusted to reduce the difference (error) between the desired target value and corresponding output. SVM offers an extremely powerful method of deriving efficient models for multidimensional function approximation and classification. Neural Networks The feedforward neural networks allow only for one directional signal flow. Furthermore, most of feedforward neural networks are organized in layers. An example of the three-layer feedforward neural network is shown in Figure1. IN P U T S H I D D E N L A Y E R S O U T P U T L A Y E R